The present invention is related to image compression, and in particular to variable length coding of a series of quantized transform coefficients of a block of image data.
Digital video compression ranges from coding still video/image to coding moving video for photographic, broadcasting, streaming, and conferencing applications. Modern transform based techniques include dividing an image into blocks, transforming the blocks of image data into transform coefficients, quantizing the coefficients, ordering the coefficients along a path, e.g., a zig-zag path on the two-dimensional transform domain, and encoding the series of quantized coefficients using a variable length coding method.
Two-dimensional variable length coding (2D-VLC) includes collecting or assuming the statistics of two dimensional block transform coefficient events that are each a run of the most-likely-to-occur amplitude, e.g., 0, followed by another amplitude. The coding includes assigning variable length codes, e.g., optimal codes such as Huffman codes or Arithmetic codes, to each event. In the description herein, 0 is assumed to be the most-likely-to-occur amplitude. The collecting or assuming statistics includes tracking the quantized non-zero-valued coefficient amplitudes and the number of zero-valued coefficients preceding the non-zero amplitude, i.e., tracking the runlengths of zeros which precede any non-zero amplitude along a specified path, e.g., a zigzag scan path for a block of coefficients, e.g., an n by n coefficient block.
Denote by Sij the likelihood expressed, for example, as a relative number of occurrences of an amplitude of i, i=1, 2, . . . occurring after a run of j 0's, j=0, 1, 2, . . . IN 2D-VLC, a variable length code such as an optimal code is assigned to each of the events that have such an Sij, with the most-likely-to-occur element—typically S10 for the case of encoding a block of transform coefficients in transform coding—having the shortest number of bits, and the least occurring event coded using the longest number of bits. The results of such coding may be tabulated as a table—a 2D-VLC table. Such a table provides the codeword, denoted Cij, used to encode the event of the combination of j consecutive 0-valued coefficients followed by a single non-zero coefficient of amplitude i, j=0, 1, . . . and i=1, 2, . . . .
2D-VLC is used in common transform coding methods such as JPEG, MPEG1, MPEG2, ITU-T-261, etc., as follows. For motion video, an image is divided into blocks, e.g., 8 by 8 or 16 by 16 blocks. Each image is classified as interframe or intraframe. Interframe images are typically post motion compensation. The blocks of the image are transformed and the transform coefficients are quantized. The quantized transform coefficients are then coded along a specified path according to a 2D-VLC table. Typically, interframe and intraframe images have different 2D-VLC tables. The DC component is typically separately encoded. Furthermore, the 2D-VLC table may be truncated so that the least frequently occurring events use an escape code followed by a fixed length code. A special “EOB” code is used to indicate the end of block when all remaining coefficients are zero.
Still images are similarly encoded, e.g., in the same manner as an intraframe image for motion video.
FIG. 1 shows how a table lookup may be used to implement a 2D-VLC scheme. Prior to the table look up, the runlength of zero amplitudes preceding any non-zero amplitude and the non-zero amplitude are determined. The table look up uses a 2D table for those likely events encoded using variable length encoding. An escape code together with a fixed length codes is used for relatively less likely-to-occur combinations
The advantage of 2D_VLC is that both the position of each non-zero-valued coefficient as indicated by the runlength, and the quantized amplitude value are coded simultaneously as a pair using one 2D-VLC table. This may result in shorter codes, i.e., codes that use fewer bits than using separate VLC tables for each non-zero-valued coefficient and for its amplitude.
Because of the widespread use of image coding, many patents have been issued on different forms of VLC. U.S. Pat. No. 4,698,672 issued Oct. 6, 1987 to Wen-hsiung Chen, one of the inventors of the present invention, for example described one form of a two-dimensional variable length coding method.
Extensions and variations to the common 2D-VLC method are known. For example, the ITU H.263 compression standard defines one such variation sometimes called three-dimensional VLC (3D-VLC). See PCT patent publication WO 9318616 published Sep. 16, 1993 titled PICTURE DATA ENCODING METHOD and also the ITU-T H.263 standard. In 3D-VLC, each symbol (“event”) is a triplet (LAST, RUN, LEVEL) that includes: LAST, a binary flag that indicates whether or not the current non-zero amplitude-value is the last non-zero coefficient in the block, RUN, the run-length of zero-value coefficients that precede the current non-zero amplitude, i.e., the number of zeroes since the last non-zero coefficient amplitude, and LEVEL, the current non-zero coefficient amplitude value. Thus, there is no need for a separate EOB codeword; whether or not the non-zero coefficient is the last one is incorporated into the event.
FIG. 2 shows how a table lookup may be used to implement 3D-VLC.
One deficiency of 2-D VLC methods is that every non-zero-valued coefficient needs to be accompanied by one runlength to identify its position. In block based transform coding, it may occur that there are a number of consecutive non-zero-valued coefficients along the predetermined coding path. This may especially occur in intraframe coding and high bit rate interframe coding. The 2D-VLC method requires a separate runlength code, e.g., C10, C20, C30 . . . , etc., for each of the consecutive non-zero coefficient. Thus there is a need in the art for a method that provides for efficiently encoding a sequence of consecutive non-zero coefficient values.
U.S. patent application Ser. No. 10/342,537 to inventors Chen et al., filed Jan. 15, 2003 and titled AN EXTENSION OF TWO-DIMENSIONAL VARIABLE LENGTH CODING FOR IMAGE COMPRESSION describes what is called the “Extended 2D-VLC Method” that includes encoding repetitions of some non-zero coefficient values. U.S. patent application Ser. No. 10/342,537 is incorporated herein by reference, and the methods described therein are each and collectively called the “Extended 2D-VLC Method” herein.
One variant of the Extended 2D-VLC method provides codes for all the possible amplitude variations of consecutive coefficients that follow a set of zero-valued coefficients. This effectively reduced the runlength to 1 for all cases. The difficulty of this approach is that there are enormous numbers of patterns that can be generated from the amplitudes of consecutive coefficients. For example, with 32 quantization levels as defined in many common video coding standards, there are in the order of 32n patterns that can be generated from n consecutive coefficients. As such, in practical implementation, only a limited number of most frequently appearing non-zero amplitude values such as 1 and 2 and, a limited number of lengths of consecutive non-zero-values, such as 3 or 4 consecutive values are regrouped for pattern matching.
Thus, there still may be inefficiency in using the called “Extended 2D-VLC Method’ in practice.
The above-referenced Basic Hybrid VLC Method of U.S. patent application Ser. No. 10/869,229 was invented as a result of the observation that there is an additional statistic that may have been overlooked in prior art variable length coding methods. Non-zero-valued quantized transform coefficients along the path tend to cluster more in the low frequency region and are more scattered in the high frequency region. That is, runs of non-zero values are more likely to occur in lower spatial frequencies than in higher spatial frequencies. The Basic Hybrid VLC Method encodes position and amplitude of transform coefficients separately to take advantage of this clustered nature of coefficients in the low frequency region and scattered nature in the high frequency region. Two types of runlength coding schemes are used. The Basic Hybrid VLC Method further takes advantage of the likelihood of having several consecutive non-zero-valued coefficients in the clustered low frequency region. In such a case, a single runlength would be used for the run of non-zeroes instead of a number of runlength codes for each of the different amplitudes, as would occur with traditional 2D-VLC.
In one embodiment, the Basic Hybrid VLC Method includes establishing a breakpoint along the path of the ordering of the coefficients to identify a first, e.g., low-frequency region and a second, e.g., high frequency region. The encoding of low frequency coefficients includes coding the positions of consecutive non-zero-valued coefficients and consecutive zero-valued coefficients using runlength coding methods of the first kind and second kind, respectively. The method further includes coding the amplitude values of the non-zero-valued coefficients in runs of non-zero-valued coefficients in the first region. The method further includes coding the runs of non-zero-valued coefficients using a runlength coding method of the third kind in the second, e.g., high-frequency region. The method further includes coding the amplitude values of the non-zero-valued coefficients in the second region. Thus, different runlength coding methods are used in the low frequency and high frequency regions. At most, five independent coding tables may be used: two runlength and a first amplitude table for low frequency coefficients and one runlength and a second amplitude table for the high frequency coefficients. The code lengths and code tables may then be shorter than that of a typical 2D-VLC table.
The above-referenced Basic Hybrid VLC Method still may be improved by looking more deeply into the nature, e.g., distribution of the coefficients in the regions. In particular, it is known that motion compensation techniques are improving, such that there may be significant runs of zero-values, and that non-zero quantized coefficient values may be dominated by a few relatively low amplitude values such as 1 and 2. Furthermore, in the high frequency region, there frequently may only be non-zero coefficients of value 1.
Thus there is still a need in the art for a method that improves on the above-referenced Basic Hybrid VLC Method by efficiently taking into account the higher likelihood of only non-zero value-1 coefficients in the high frequency region of an ordered series of quantized transform coefficients of blocks of an image.
One or more patents describing some existing 2D-VLC coding methods have recently been the subject of patent litigation. Thus, there is a need in the art for alternate methods that can replace commonly used 2D-VLC methods that have been the subject of such litigation.